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DETAILED ACTION 

Specification 

The disclosure is objected to because of the following informalities: the 
paragraph 14 in the column 2 cites "very good probability" which is indefinite. 
Appropriate correction is required. 

The disclosure is objected to because of the following informalities: there is an 
inconsistency of citing the subset "Ik" in the numerous palaces which is recited 'V in the 
column 2, the paragraph 43 as well. 

Appropriate correction is required. 

Drawings 

The drawing is objected because the photographs must be of sufficient quality so 
that all details in the photographs are reproducible in the printed patent. 
The informal drawings are not of sufficient quality to permit examination. Accordingly, 
replacement drawing sheets in compliance with 37 CFR 1 .121(d) are required in reply to 
this Office action. The replacement sheet(s) should be labeled "Replacement Sheet" in 
the page header (as per 37 CFR 1 .84(c)) so as not to obstruct any portion of the 
drawing figures. If the changes are not accepted by the examiner, the applicant will be 
notified and informed of any required corrective action in the next Office action. 
Applicant is given a TWO MONTH time period to submit new drawings in compliance 
with 37 CFR 1 .81 . Extensions of time may be obtained under the provisions of 37 CFR 
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1 .136(a). Failure to timely submit replacement drawing sheets will result in 
ABANDONMENT of the application. 

The photographs in Figure 3 are not acceptable to be examined. 

The drawing is objected to because of the following informality: The item 1 1 in 
the Fig2 explains determining a subset as the condition "d(i,k) T for c", but it is not 
intelligible and the examiner assumes "d(l,k1) < T for ie Ik". . 

Appropriate correction is required. 

Claim Objections 

Claim 1 is objected to because of the following informalities: it cites very good 
probability which is indefinite. Appropriate correction is required. 
Claim 4 is objected to because of the following informalities: The last sentence is 
incomplete and ends with comma. Appropriate correction is required. 

Claim Rejections - 35 USC §112 

Claim 2 is rejected under 35 U.S.C. 112, first paragraph, as failing to comply with 
the enablement requirement. The claim(s) contains subject matter which was not 
described in the specification in such a way as to enable one skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and/or use the invention. 
It is not clear how Monte Carlo method is specifically used even if the specification 
discloses the reference ("Robust regression method for computer vision: a review", 
Rosenfeld). 



Application/Control Number: 10/532,282 Page 4 

Art Unit: 4192 

Claim Rejections - 35 USC § 101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 

Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. 

In summary, the claimed methods are not tied to a particular machine or 
apparatus. Additionally, none of the recited steps transform a particular article into a 
different state or thing. 

Accordingly, the recited method is nonstatutory subject matter. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claim 1 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bradley et 
al. ("Refining Initial Points for K-Means Clustering", Microsoft Research Technical 
Report, May 1998, pp. 3-4) in view of Paul E. Green ("Analyzing Multivariate Data", The 
Dryden Press, 1978, pp.427-428) and Jain et al. ("Algorithms for Clustering Data", 
Prentice Hall, 1988, pp. 96-1 01), hereafter referenced as Bradley and Green and Jain 
respectively. 
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Regarding claim 1 , Bradley discloses random drawing of p candidates (Random 
Subsamples in the Algorithm Refine, page 3) from the set of key images, p being 
calculated in such a way as to obtain a very good probability of drawing a key image of 
a prevalent shot, calculation of the cost C for each candidate (Distortion in the 
Algorithm Refine, page 3), dependent on the distance from the key images of the set to 
that of the candidate, the distance relating to the signatures, and selection of the 
candidate (k I) minimizing the cost C (ArgMin{Distortion} in the Algorithm Refine, page 
3). However Bradley fails to disclose determination of a subset (Ik) from among the set 
of key images such that the key images forming the said subset have a distance from 
the candidate less than a threshold T, deletion of the key images of the subset (Ik) to 
form a new set of key images for at least one new random draw and determination of a 
new seed according to the previous 5 steps, and determination of a seed (k2) from 
among the key images of the subset (Ik) such that it minimizes the cost function C for 
this subset. However the examiner maintains that it was well known in the art to provide 
the above as taught by Green and Jain. 

In a similar field of endeavor Green discloses determination of a subset (Ik) from 
among the set of key images such that the key images forming the said subset have a 
distance from the candidate less than a threshold T (all object within prespecified 
threshold are grouped) and deletion of the key images of the subset (Ik) to form a new 
set of key images(Once points enter a cluster they are removed from further 
processing) for at least one new random draw and determination of a new seed 
according to the previous 5 steps (the process is repeated for the process is repeated 
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for the uncluttered points, and so on), as disclosed in Sequential Threshold at Page 
428. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bradley by specifically providing determination of 
a subset and deletion of the key images of the subset as taught by Green, for the 
purpose of reduction of computation cost by optimizing sequentially, one seed at a time, 
instead of optimizing all seeds in parallel. 

In a yet another similar field of endeavor Jain discloses determination of a seed 
(k2)from among the key images of the subset (Ik) such that it minimizes the cost 
function C for this subset (compute new cluster centers as the centroids of the clusters) 
as disclosed in Algorithm For Iterative Partitional Clustering Step 3 at page 97. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify Bradley and Green by specifically providing 
determination of a seed (k2) from among the key images of the subset (Ik) such that it 
minimizes the cost function C for this subset, as taught by Jain for the purpose of 
optimizing a centroid from a cluster. 

Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable over Bradley 
and Green and Jain in view of Palmer ("Density Biased Sampling: An Improved Method 
for Data Mining and Clustering", Proceedings of ACM SIGMOD International 
Conference on Management of Data, 2000), hereafter referenced as BGJ and Palmer. 

Regarding claim 3, BGJ discloses everything claimed as applied before (see 
claim 1 ), however they fail to disclose wherein the key images are weighted, as regards 
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their signature, as a function of the length of the shots of the video sequence that they 
characterize and in that the random draw is biased by the weight of the key images. 
However the examiner maintains that it was well known in the art to provide the above 
as taught by Palmer. 

In a similar field of endeavor Palmer discloses wherein the key images are 
weighted, as regards their signature, as a function of the length of the shots of the video 
sequence that they characterize and in that the random draw is biased by the weight of 
the key images ((iii)the sampling is biased by group size, page 3). 

Therefore it would have been obvious to one of ordinary person in the art at the 
time the invention was made to modify BGJ by providing the biased sampling taught by 
Palmer for the purpose of getting more realistic sampling in the video summary 
application. 

Claim 4 is rejected under 35 U.S.C. 103(a) as being unpatentable over BGJ in 
view of Foote (US patent 6,774,917), hereafter referenced as Foote. 

Regarding claim 4, BGJ discloses everything claimed as applied before (see 
claim 1). Bradley further discloses wherein the cost C is dependent on the quadratic 
distances (the sum of squared distances of each data point to nearest means) between 
the signature of the candidate and those of the key images of the subset. However BGJ 
fails to disclose that T is the standard deviation of the distribution of the distances of the 
key images of the set from the candidate. However the examiner maintains that it was 
well known in the art to provide the threshold T as a standard deviation of the 
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distribution of the distances of the key images of the set from the candidate as taught by 
Foote. 

In a similar field of endeavor Foote discloses Methods and Apparatus for 
Interactive similarity searching, Retrieval, and Browsing video. Specifically Foote 
discloses that a multiple of standard deviation is used as threshold(T) in the figure 18 
and the column 5, line 36-39. 

Therefore it would have been obvious to one of ordinary person in the art at the 
time the invention was made to modify BGJ by providing the threshold as a standard 
deviation, as taught by Foote, for the purpose of detecting class membership effectively 
using statistics. 

Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over BGJ in 
view of Wan et al. ("A multiresolution color clustering approach to image indexing and 
retrieval", Proceedings of 1998 IEEE International conference, pp. 3705-3708), hereafter 
referenced as Wan. 

Regarding claim 5, BGJ discloses everything claimed as applied before (see 
claim 1), however they fail to discloses wherein the signature of an image relates to the 
dominant color. However the examiner maintains that it was well known in the art to 
provide the above as taught by Wan. 

In a similar field of endeavor Wan discloses the dominant color as a feature as 
disclosed in the sub clause 3.1 , which is equivalent to signature of an image relates to 
the dominate color. 
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Therefore it would have been obvious to one of ordinary person in the art at the 
time the invention was made to modify BGJ by providing the dominant color as a feature 
as taught by Wan for the purpose of quickly differentiating clusters at the coarse 
resolution. 

Claim 6-8 and 10 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over BGJ in view of Vailaya et al. ("Video Clustering", Technical Report MSUCPS:TR96- 
64, Michigan State University, 1996), hereafter referenced as Vailaya. 

Regarding claim 6, BGJ discloses everything claimed as applied before (see 
claim 1). However BGJ fails to discloses shots of a sequence of video images, the 
sequence being split into shots , a shot being represented by one or more key images, 
at least one signature or attribute being calculated for the key images, comprising a 
phase of partitioning the key images on the basis of a comparison of the attributes of 
the key images, comprising a phase of initialization for the selection of at least two key 
images or seeds on the basis of which the comparisons for the grouping are performed. 
However the examiner maintains that it was well known in the art to provide the above 
as taught by Vailaya. 

In a similar field of endeavor Vailaya discloses the sequence being split into 
shots (shot detection), shot being represented by one or more key images (keyframe 
extraction), signature or attributefteature extraction), and comprising a phase of 
partitioning the key images (clustering) in Figure 1 . 

Therefore it would have been obvious to one of ordinary person in the art at the 
time the invention was made to modify BGJ as taught by Vailaya by providing the shot 
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detection, key-image extraction, feature extraction, and the clustering for the purpose of 
video summary or indexing. 

Regarding claim 7, BGJ and Vailaya disclose everything as claimed as before 
(see claim 6). In addition Jain also discloses wherein the partitioning phase implements 
an algorithm of the K-means or K-medoid type (K-means type algorithm at the page 99) 
by citing that K-means type algorithms converge rapidly. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to further modify BGJ by specifically providing wherein the 
partitioning phase implements an algorithm of the K-means or K-medoid type as taught 
by Jain, for the purpose of faster convergence of clustering. 

Regarding claim 8, BGJ and Vailaya disclose everything as claimed as before 
(see claim 6). In addition Green also discloses wherein the initialization and partitioning 
phases are iteratively repeated (the process is repeated for the process is repeated for 
the uncluttered points, and so on), the key images of the most compact cluster obtained 
in the previous iteration being eliminated from the set processed at this previous 
iteration so as to provide a new set on which the new iteration is performed (Once 
points enter a cluster they are removed from further processing), as disclosed in 
Sequential Threshold at Page 428. 

Therefore it would have been obvious to one of ordinary person in the art at the time the 
invention was made to modify BGJ and Vailaya by providing the Sequential Threshold 
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Method as taught by Green as a partitioning for the purpose of reduction of the 
computation. 

Regarding claim 10, BGJ and Vailaya disclose everything as claimed as before 
(see claim 6). Also it discloses everything for claim 10 because Method of selecting 
shots of interest, these shots being prevalent in the video sequence, characterized in 
that it implements implementing the method according to Claim 6, the shots of interest 
corresponding to the grouping performed about the first seed selected) is a part of 
process disclosed in claim 6. 

Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over BGJ and 
Vailaya in view of Turi et al.("K-means Clustering for Colour Image Segmentation with 
Detection of K", Proceedings of the LASTED international Conference Signal and Image 
Processing, Oct 27-31 ,1998, pp. 345-349), hereafter referenced as Turi. 

Regarding claim 9, BGJ and Vailaya disclose everything as claimed as before 
(see claim 8). However BGJ and Vaiaya fail to disclose wherein the stopping criterion 
for the iterations is dependent on the number of key images not belonging to the most 
compact cluster selected or else is dependent on the averages of the intra-cluster 
distances. 

However the examiner maintains that it was well known in the art to provide the above 
as taught by Turi. 

In a similar field of endeavor Turi discloses the stopping criterion dependent on 
the averages of the intra-cluster distances (Equation 1 at the page 346). 
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Therefore it would have been obvious to one of ordinary person in the art at the 
time the invention was made to modify BGJ and Vailaya by providing the intra-cluster 
distance as a stopping criterion as taught by Turi for the purpose of compactness of the 
clusters. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to HEE-YONG KIM whose telephone number is (571 )270- 
3669. The examiner can normally be reached on Monday-Thursday,8:00am-5pm EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Jeff Harold can be reached on 571-272-7519. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
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For more information about the PAIR system, see http://pair-direct.uspto.gov. 
Should you have questions on access to the Private PAIR system, contact the 
Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like 
assistance from a USPTO Customer Service Representative or access to the 
automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272- 
1000. 

Examiner Hee-Yong Kim 
H.K 

/Jefferey F Harold/ 

Supervisory Patent Examiner, Art Unit 4192 



